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ABSTRACT 

This paper documents the development of a conceptual 
framework to ensure that the necessary steps are undertaken for 
effective policy im-'lementation. These steps include the articulation 
of policy, the translation of policy into procedures, the 
operational izing of those procedures into rules, and then both 
internal (formative) and external (summative) evaluation. The 
flexible framework incorporates an adapted versiun of Stafford Beer's 
model of recursive systems to illustrate the hierarchical but 
interdependent levels or groups of people involved in a statewide or 
provincial educational system, where policy is often initiated at one 
level to be carried out by representatives at another level. Not only 
does such a framework assist in the communication of policy, but Uso 
it is developed to encourage th» ongoing evaluation of policy at 
different levels of the system. References and figures are included. 
(Author/TE) 
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POLICY DOCUMENTATION AND ASSESSMENT 
A CONCEPTUAL FR/^MEWORK 



Abstract 

The purpose of the paper is to document the development of a 
conceptual framework in order to ensure that the necessary steps are 
undertaken for effective policy implementation. Those steps include the 
articulation of policy, the translation of policy into procedures, the 
operationalizing of those procedures into rules, and then both internal 
(formative) and external (summative) evaluation. The flexible framework 
incorporates an adapted version of Stafford Beer's model of recursive 
systems to illustrate the hierarchical but interdepenoent levels or 
groups of people involved in state-wide or provincial educational system, 
where policy is often initiated at one level to be carried out by repre- 
sentatives at another level. Not only does such a framework assist in 
the communication of policy, but also it is developed to encourage the 
ongoing evaluation of policy at different levels of the system. 



POLICY DOCUMENTATION AND ASSESSMENT: 
A CONCEPTUAL FRAMEWORK 



Introduction and Overview 

This paper develops a conceptual framework for policy documentation 
and assessment. Incorporating an adapted version of Beer's (1979) model 
of recursive systems, the framework attempts to facilitate both an 
understanding and the use of the policy-making process — a cyclic 
process initiating with policy documentation or description and ending 
with policy evaluation. 

The concept of policy, as with many other concepts in management 
science, is usually described in vague and nebulous terms, if at all* 
Perhaps the lack of a commonly understood and accepted definition ould 
be the reason attributed lo policy ofren not being defined in literature 
dealing with policy formation, implementation, or evaluation. In fact, 
Cuba (1984) he s culled eight definitions of policy from the literature, 
ranging from general description (such as, "any governing principle, 
plan, or course of action") to specific prescription, and being defined 
as a process as well as a product. Cuba also discusses how the selected 
definition determines what Weick (1979) would refer to as the policy- 
maker's "enacted environment" — i.e., how a problem is perceived, 
initially, affects its solution and/or analysis. What app<>ars to be 
implied by Cube and perhaps even more clearly expressed by Japanese 
management proponents, is that if such integral terms are not clarified 
for members of the organization at the outset, the attainment of 
objectives becomes difficult to assess. This concern is not new 
(Elboim-^Dror, 1970); perhaps it might be considered on a more general 
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level, as Indicated by March and Simon (1958): intangible, unclear, and 
vaguely defined objectives impair the effectiveness of a system. 

A lack of clarification of policy statements and, accordingly, the 
domain of policy statements not only renders assessment of policy 
attainment difficult, but also may cause confusion over which group of 
individuals is accountable for policy accomplishments. As has been 
pointed out by Milward (1982), the raore widesweeping the policy, a 
greater number of organizations can be involved. Moreover, policy can 
originate, on the one hend, from outside an organization, and 
particularly for those not-for-profit publically-supported social- 
serving organizations (Newman & WaLlendar, 1978), such as schools. On 
the other hand, policy can be initiated from inside the organization. 
To complicate the issue further, policy can be directed to the entire 
organization or only to one of the three subsystems comprising an 
organization, to what Parsons (1956) referred as the institutional, 
managerial, and operational subsystems. (To avoid ambiguity, these 
subsystems have been renamed in this paper as the strategic, 
administrative, and operational subsystems.) 

The possible confusion that could occur, then, at one or more of 
the policy-making, policy-implementing, and policy-evaluating stages by 
these stakeholders, who may or may not be the same group of individuals, 
provides the rationale for the development of a conceptual framework for 
articulating policy. Hence, to promulgate the notion that policy can 
involve, potentially, a wide range of individuals both within and 
outside of the organization, the suggestion here is that policy be 
viewed in a recursive model, similar to that as first proposed by Beer 
(1979), and later adapted by Rees (1983). First, to highlight that the 



model is Intended Co provide flexlblll«-y and to convey a loose-coupling, 
the following definition of policy is used to focus this thesis: "a 
proposed course of action, or the guideline for decision-making that is 
intended to enable the institution to attain its goals." Second, rather 
than policy statements referring both to the system as a whole and to 
each of its subsystems, the model, in essence, disaggregates sweeping 
policy statements into increasingly more specific guiding principles of 
policy, procedures, and rules. And third, each of these three 
guidelines, correspondingly, is mapped onto the strategic, 
^administrative, and operational subsystems, the three subsystems which 
constitute any viable system/organization* 

This conceptual framework has several Dteworthy attributes* 
One, the model recognizes the interdependence of policy development, 
implementation, and evaluation. Two, the model can act as a 
communication device to identify these groups that are responsible for 
different aspects of the policy-making process , (from initiation, to 
enactment, to monitoring, and also for assessment). And finally, the 
framework acknowledges the differences in evaluation that do and should 
continue to take place on a system-wide basis: the internal subsystem 
assessment for formative or developmental purposes, and the external 
evaluation to validate subsystem or system goal achievement for the 
purpose of accountability to an external public. Hopefully, the 
benefits of such a framework should become immediately apparent to 
interested in managing effective policy — people existing at each and 
every level of a policy-making hierarchy. 



The Conceptual Framework 

The conceptual framework is developed in detail in this section of 
the paper. The objective is to arm the practitioner with sufficient 
knowledge of the model and how it was formulated in order to encourage 
its appropriate use, and in the wide variety of situations for which it 
was intended. 

In a proposal for research of the Ontario Government's educational 
evaluation policy, Wilson and Rees (1985) identified at least five 
levels of policy-making for the Ontario educational system. They are 
the Ministry of Education, the school board or district, the school 
(either inclusive or exclusive of departments or other relevant 
subanits), the classroom, and the student. Educational policy could be 
developed and, indeed » undoubtedly has been developed for each of these 
five levels discretely, and for more than one of these levels 
conrurrently, either by officials internal to that particular level or 
by Chose at a higher level of authority. 

If the intent of policy-making is to ensure that the organizational 
goals are being met relatively efficiently and effectively, it seems 
sensible to carry out some assessment or evaluation of the policy. 
Accordingly, documentation of policy must be the first step of any 
evaluative process. Figure 1, below, demonstrates an initial attempt at 
identifying what policy has been developed. 

[Insert Figure 1 here] 

This first diagram offers more than Just a format for documenting 
policy: it distinguishes among those levels within the educational 



system — the leveKs) at which the policy originated from the level(s) 
at which the policy was intended* For example, policy concerning the 
reporting of student progress is obviously directed to the student* 
But the initiators could be one of more of the following: teacher (who 
represents the classroom level), principal (representing the school), 
superintendent (representative of the school board), and so forth. The 
diagram illustrates, moreover, that policy can be micro and/or macro in 
perspective, and more overarching as it relates to higher hierarchical 
levels within a provincial educatioi.il system. In these respects. 
Figure 1 offers a mechanism for documenting more than just a statement 
of policy. 

Figure 1 has a serious shortcoming, however. It portrays each 
level within the educational systeu strictly as an independent, rather 
than as a dependent, unit. More appropriately, the cells should be 
viewed as interrelated. For example, a board^s policy relating to the 
documentation of student progress couJJ be completely separate from the 
school's policy. But a school's policy could be the result of its 
board's policy. Both the users and the appraisors of the policy(ies) 
should be cognizant of that distinction. The policy evaluators, in 
particular, should be concerned whether each inter-level policy 
statemant is consistent with its higher level one. 

Figure 2 is an attempt to overcome that major flaw identified in 
the initial diagram. First of all. Figure 2 depicts the educational 
system as a triangle, as Anthony (1965) proposed, reinforcing the 
perspective that the base of the triangle, the operational core, is the 
raison d'^etre of any viable organization (Emery & Trist, 1969). 
Secondly, the concept of the hierarchical yet interdependent levels is 
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included by showing the nested or embedded levels ot responsibility and 
authority within the overall educational system. This model could 
easily be expanded or contracted to Illustrate the chain of command, the 
different hierarchical levels, that are Involved In each phase of policy 
Initiation and policy enaction. The diagram also usefully shows the 
different levels of policy, from complex policy that Initiates from 
outside an Institution to a more simple policy that both originates and 
Is Intended for the one and the same hierarchical level within an 
organization. Overall, then, the diagram displayed In Figure 2 
demonstrates a flexibility and verlslmulltude that Is missing from 
Figure 1. Figure 2 is preferable, not only because it provides a means 
for documenting policy, but also because it has the capacity to pinpoint 
both how and which Inter and Incra-lnstltutlonal levelCs) are Involved 
In a particular policy. 



[Insert Figure 2 here] 



Despite these merits. Figure 2 neglects to show a connection 
between policy Initiation and policy enactment. A linchpin Is required, 
one which provides policy Interpretation and support (Fullan & Park, 
1981). 

Parsons (1956) was the first to propose that organizations as 
systems have three , not two, subsystems which are Interdependent , not 
dependent. While each subsystem carries out certain activities or 
functions In a relatively Independent manner, the subsystems, 
collectively, require some coordination via an Intermediary subsystem to 
attain a system'^s overarching goals. 
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Each of the subsystems has a different purpose. The first 
subsystem is the institutional subsystem, having both an external and an 
internal focus. Its external focus is aimed at defending, legitimating, 
and maintaining the organization, its goals^ and its domain with 
respect to its environment to ensure the viability (survival and growth) 
of the organization. Its inward focus deals primarily with policy 
formulation, providing direction necessary for overall organizational 
goal attainment. The second subsystem, the one in the middle, was 
labelled by Parsons as the managerial subsystem, carrying out all the 
necessary functions to integrate the policy initiators with the policy 
implementors. Those activities are coordinative in nature, including the 
monitoring and control aspects, and allocating and scheduling the 
resources provided by the institutional subsystem. And the third 
subsystem, the operational subsystem, is concerned with policy 
implementation, using the resources assigned to transform 
organizational intention into ac tiom (Because several contentious 
meanings exirt for both the words 'institutional^ and ^managerial^, the 
words ^strategic'' and ^administrative^ are used in the remainder of this 
paper to replace the words ^InstitutionaT and ^managerial^) 

The direct application of Parsons' concepts to policy'-making, then, 
reveals that policy must be handed down, by means of a communication 
process: from the apex of the organization, (those in the strategic 
subsystem); by way of the middle managers, (those in the administrative 
subsystem); to those ''front-line'' individuals, (those involved directly 
in operations). Furthermore, Stoner (1978) has suggested that 'policy 
devolution' process occurs in the following three distinct formats of: 
(1) policy, which is duly broken down into (2) procedures, which are 
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made even more specific in the form of (3) rules. 

Figure 3 offers a means of showing both the one-to-one mapping and 
the processes occurring in these subsystem?-. First, the strategic 
subsystem formulates policy. Then the administrative subsystem 
interprets the policy into procedures, the standard ways and means to be 
followed* And third, the operational subsystem rephrases the procedures 
into even more specific guidelines known as rules, the outline of the 
uses of the resources in order to implement policy. 



To summarize, the five hierarchical but interdependent levels of 
any provincial educational system, as shown in Figure 2, can each be 
considered as a system In itself. And each of these levels is comprised 
of three subsystems, as revealed in Figure 3. What remains, then, is 
that these two diagrams be combined. But first, one question requires 
answering: How are these levels interrelated or, more to the point, 
where (in which subsystem) is one educational level nested within its 
hierarchical level? Beer (1979), in his ongoing study of cybernetics as 
it applies to open systems theory, has grappled with a similar problem. 
As a solution, he developed a theory of recursion. 

Beer noted that a system becomes increasingly process-specialized 
or functionally-differentiated over time. Rather than adding more 
discrete subsystems as others such as Miller (1978) proposed, Beer 
contended that the model must highlight the interrelated nature of these 
ongoing activities. Although he believed that each viable system was 
comprised of five, not the more commonJy-accepted three, subsystems, he 



[Insert Figure 3 here] 
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conceptualized chat Che embedding occurs within the lowest tier, the 
operational subsystem. Within that operational subsystem nests another 
complete system. Similarly^ a third system is embedded within the 
second system'^s operational subsystem. This recursivity» Beer argued» 
rould continue ad inflnltuB, depending on the number of levels or 
information available and the sophistication of the large-scale system. 
This ''systems within systems'" concept is known as Beer^s theory of 
recursion. It recognizes, as no other known model does, both the 
interdependence of systems (or subsystems, depending upon the focus or 
unit of analysis used) and the interdependence of functions at the 
hierarchical levels of decision-making that occur within the 
implementation staje of any process. 

Figure 3, a concatenated versl n of Beer^s model where the five 
subsystems have been reduced to three in conformance with the Parsonlan 
norm, can be used to represent any system. In addition. Figure 4 
illustrates Beer'^s theory of recursive systems, using only two systems 
in order to demonstrate where (in which of the three subsystems) the 
nesting occurs. More accurately, Beer would contend that the latter 
diagram portrays a system which is more complex than that of Figure 3. 
The more complex model contains two levels of recursion. The adapted 
version of Beer^s model was deve.loped by Rees (1983). In a study using 
this model, she demonstrated the hierarchical yet interactive nature of 
manpower training at the federal, provincial (Manitoba), and local 
levels. 



[Insert Figure 4 here] 
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The final diagram, Figure 5, combines the concepts portrayed in the 
previous drawings to illustrate how an educational policy in Oncario 
might be captured. It tries to delineate the key assumptions that we.e 
r:i^.de thus far: 

1. A provincial educational system if comprised of at least five 
hierarchical levels; 

2. Each of these levels can be considered as neither completely 
independent nor dependent of its hierarchical one, but rather 
incerrelated, with different levels of responsibility and authority both 
between or among le as well as within levels; 

3. Each of these levels is viewed as a syste-n containing three 
functionally-differentiated out interdependent subsystems, known as 
the s^.rategic, administrative, aiid operational subsystems; 

4. Each of these levels is nested within another level, a higher order 
system, emHedded witnin that latter system'^s operational subsystem; 

5. The -olicy-making process includes the functions of policy 
initiation, policy translation/interpretation, policy implementation, 
and policy evaluation, with potentially different personnel responsible 
for each of these activities. 

6. Each cf these levels within the educational system involved in a 
pai*:icular policy must carry out the policy statement in a consistent 
fashion, through the use of procedures and then rules. 

(Insert Figure 5 here] 

Figure 5 offers a flexible conceptual framework for dealing with 
policy documentation and assessment. Equally well, it represents the two 
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situations where: 

a) Policy is externally imooped on an organization; a higher order 
system formulates policy for another educational level* 

b) Policy is imposed from within an institution; an organization 
develops policy for itself* 

Figure 5 shows, moreover, that policy can have a macro and/or a 
micro intent, depending on the complexity of the issue and the 
'^distance'^ between the policy originators and the policy implemeutors* 
Finally, if policy assessment is to occur r't all realistically, first 
policy^ procedures, and rules must be documented in order to traced 
from the Jevel of origination, through to the intermediary system(s) an-l 
subsystems, to the implementation stage* 

The Benc'f it of this Conceptual Framework for Evaluation 

It was stated at the beginning of ^he paper that this model was 
intended not only to aid in the explicit documentation of policy, but 
also to provide a mechanism for policy evaluation* Ironically, 
throughout North America, organizations are reiterating such phrases as 
"the search for excellence" and "increased accountability*" Yet without 
a means of assessing how to determine the degree to which the policies 
are working and the degree to which they have been implemented, no 
decision about success can be made that might rationally lead to 
improvement* 

Unfortunately, a wariness seems to exist with respect to 
evaluation* It is something that more people like to talk about than to 
do* Wilson (1975) has noted that different parties associated with a 
particular policy may have dl' «ient expectations* Often, the results 
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of one assessment are used for another purpose* Perhaps the reluctance 
to take an active part In evaluation occurs because the reason for the 
evaluation was act stated, and neither was the policy clear, nor were 
the various groups responsible for the different aspects of the process 
(external and/or Internal to the organization) known. 

Scriven (1967) has made a useful distinction between the goali of 
monlr.orlng for policy Implementation and the value of policy. He calls 
the first "formative evaluation" and the second "summatlve evaluation**' 
The conceptual framework In this paper, by means of the hierarchical but 
Interdependent systems, helps to differentiate, visually, the two types 
of evaluation that should be occurring. Those educational levels which 
are Involved In the policy Initiation to the policy Implementation 
stages must take part In the formative evaluation of policy. But the 
next higher order system must be responsible for the summatlve 
evaluation. 

For example, a teacher having Imposed a particular testing policy 
on the students within the classroom should be able to assess the 
results Individually by student and collect' ^ely for the class. If the 
testing policy devolved from the principal, both parties should assess 
how consistently and how well each of their policies are working and the 
degrees of Implementation. But the next higher level outside of this 
policy sphere, the school board, should be the one responsible for 
determining whether that school's policy was worthwhile. 

By adhering to the conceptual framework, evaluation of policy 
should be facilitated. The policy statements are clearly articulated 
and are disaggregated not only Into different levels of hierarchical 
systems, but also Into different subsystems. Thus, depending upon the 
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complexity of the policy statement and the point of initiation, at least 
five levels within the provincial educational system could be involved 
in formative evaluation* 

Hence, the conceptual framework can be combined with evaluation 
theory to make a complete strategy for the evaluation of any policy. 
One, evaluation requires a clear statement of intent; in other words, 
unless policy is documented in a way that is understood by its users, 
evaluation is irrelevant and impotent. Two, the complexity of policy, 
and especially wide-scale social policy, may encompass many educational 
levels and different groups of people. Communication of policy must be 
prevalent and consistent through the policy origination, 
interpretation, and enactment stages of the policy-making process. And 
three, both formative and summative evaluation should occur at each of 
these levels, in order to assess the degree of policy implementation and 
the merit of the expressed policy. 

Summary and Conclusions 

The purpose of thie oaper was to describe the development of a 
conceptual framework that would be pertinent to both the documentation 
and the assessment of policy. Indeed, as has been noted, evaluation 
should not even occur without a preliminary articulation of policy. 
Hence, using an adapted version of Beer's model of recutsive systems, a 
framework was developed to show the interconnected steps of policy 
within a complex social system.* that of a provincial educational system. 

An adaptable schematic representation was designed to reveal the 
interdependent, but hierarchical levels that may exist within any 
provincial educational system, as well as the three distinct stages that 
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must occur — policy Initiation, policy translation, and policy 
Implementation* Each of these stages was mapped onto one of the three 
strategic, administrative, or operational subsystems. In order to 
communicate the notion that the feasibility of a policy depends on Its 
disaggregation Into policy, procedures, and rules. 

Finally^ the usefulness of this model was shown In terms of Its 
applicability to evaluation. The model tries to pinpoint not only what 
are the policy, procedures, and rules that are recursively being 
unveiled to ensure that policy Is Implemented; but also vho are the 
participants. Each level and each subsystem within each level Involved 
In a policy should be Involved In Its assessment. Consequently, the 
explicit format clarifies who Is accountable for which area of 
responsibility. What Is likely to occur before anything else however, 
Is that the Inves t/.gators may discover that not all the parts of the 
model will be In place. Perhaps that would be the most Important 
evaluation finding of all. 
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FIGURE 2 



INTERDEPENDENT LEVELS OF INVOLVEMENT IN POLICY 
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A SYSTEM WITH TWO LEVELS OF RECURSION 
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FIGURE 5 

THE RECURSIVE LEVELS OF INVOLVEMENT IN POLICY 
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